Repository navigation
fix(cli): --model resolves core's aliases instead of matching three names exactly - #546
Merged
NandhaKishorM merged 2 commits intoSep 27, 2026
Merged
Conversation
build_parser declared `--model choices=sorted(laya.DEFAULT_MODELS)`, and argparse matches choices exactly. So the flag knew three spellings while the call it feeds, `router.predict(model=...)` / `router.route(model=...)`, resolves core's whole registry: three checkpoints plus ten aliases, case- and whitespace-insensitively. `laya "..." --model en` exited 2 with "invalid choice" instead of pinning English. The flag now runs its argument through `laya.router.normalise_name` -- the same call core makes -- so it cannot fall behind the registry again, the name handed to the router is already canonical, and `auto` means the same as omitting the flag.
Owner
|
Thanks @aashish254, merged. It ships in 0.3.21. |
4 tasks done
NandhaKishorM
pushed a commit
that referenced
this pull request
Sep 29, 2026
… resolves one `laya/router.py:112 normalise_name` is core's one model resolver: trim, lower-case, map `_ALIASES`, then check `DEFAULT_MODELS`. Every other surface hands a caller's `model` string to it -- `laya/serve.py::_resolve_model`, `laya/cli.py::model_name` (#546), `laya/mcp/tools.py` (#544) -- and `examples/server.py` was the exception: it tested the stripped string for membership in a tuple of the three checkpoint names. So `laya --model laya` pinned the English checkpoint while the same value POSTed to /predict answered 422, and the caller's only notice was a validation error. Measured on main, weight-free, 82 spellings probed against both resolvers: core accepts 67, the demo accepts 9. The 60 the demo refused are every alias (`laya`, `en`, `ml`, `multi`, `typed`, `typed_decisions`, `decisions`, `default`, `laya-multilingual`, `laya-typed-decisions`), every casing (`ENGLISH`, `Laya-Multilingual`) and every padded form (`" default "`, `" ml "`). Over HTTP `{"model":"laya"}`, `{"model":"LAYA"}`, `{"model":"ml"}`, `{"model":"laya-multilingual"}`, `{"model":" default "}`, `{"model":"typed_decisions"}` and `{"model":"decisions"}` each answered 422 against a Router that would have answered any of them. `_check_model` now delegates to `normalise_name` and returns the canonical checkpoint, so what reaches the Router and what the response reports is the checkpoint rather than the spelling that named it. Blank stays "do not pin": the playground posts an empty `model` field and core's resolver raises on one, so the demo keeps its own leniency there instead of becoming stricter than core -- those two blanks are the only remaining difference between the two resolvers, and they are deliberate. Everything this server publishes about the field is read from the same two tables, so an alias cannot outlive its target and a checkpoint core gains is pinnable here with no edit to this file: `MODEL_ALIASES` is `_ALIASES` inverted and filtered to live targets, `/models` reports the aliases it never listed, both request models' field text is rendered from the lists (`BatchRequest.model` had no description at all), and the models page's checkpoint count and lead text follow the payload. README's section for this server says the field exists and where its spellings come from; before this the only way to learn it was to get a 422. Gates: `tests/test_example_server_limits.py` 66 -> 81 weight-free checks, and the probe list is built from `laya.router.DEFAULT_MODELS`/`_ALIASES` rather than typed here, so it covers spellings this file has never heard of. Two arms hold the derivation: one registers a fourth checkpoint and an alias, reloads the demo, and requires it to be pinnable by name, by alias, in the OpenAPI text and on `/models`; the other parses both lists back out of the rendered description and compares them with what the router accepts, which a hand-typed sentence would not survive. `tests/test_server_example.py` pins `/models`' key set, and the payload gained a key, so that one assertion now lists `aliases` -- additive, no existing key changed name or meaning. Verified against real weights on MPS: 11 spellings across two checkpoints, every one HTTP 200 with an identical decision per checkpoint (department "billing", urgency 1.599 and 1.003), the reported checkpoint exactly {multilingual} and {english}, and blank still agreeing with omitted. The 13-mutant battery (exact-match restored, spelling returned instead of canonical, blank raising, alias map emptied, lower-only, literal description, undocumented batch field, aliases dropped from the payload, typed page count, omit-hint gone, README paragraph dropped, no reload, hardcoded probe) is caught by at least one check per mutant and restores byte-exact. Full CI script list, 60 suites, reruns clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix(cli):
--modeltook three spellings; core has thirteenThe gap
laya/cli.py:50was:choices=is derived from core, so the names never went stale — but argparse matches achoice exactly. Meanwhile the flag's whole job is to hand a value to
router.predict(model=...)/router.route(model=...), and those resolve the argument throughlaya.router.normalise_name: trim, lowercase, then_ALIASES. Core has ten aliases for its threecheckpoints. The CLI accepted none of them, and its error message told the user those three
spellings were all that exist:
--model enis what the README's own SDK examples say to write, andlaya --lang de --model mlis a normal thing to try.
laya --predict --model en(loads the checkpoint) failed the same way,so the only workaround was spelling the checkpoint out.
This is the same drift #544 fixed on the MCP surface, through a different mechanism: not a copied
set that went stale, but exact matching where core does normalisation.
Before and after (18 spellings, argparse only)
english,multilingual,typed-decisionsen,laya,default,multi,ml,laya-multilingual,typed,typed_decisions,decisions,laya-typed-decisions,EN," Multilingual ","Typed-Decisions"— all canonicalised — andauto/AUTO, which pin nothinggpt4,"",english-ish,laya-typed, and any non-stringThe error text is core's, so it now lists both the checkpoints and the aliases, with
, or 'auto'appended:With real weights (CPU,
laya --predict --json)One refund ticket through the router's own question set, five spellings of the English checkpoint:
--modelrouting.modeldomaindifficultyis_sensitiveneeds_toolsenglishenglishchitchatenenglishlayaenglishdefaultenglishENenglish0 decision changes — the alias path and the name path are the same call. The routing path
(no download) was checked the same way:
--model en/ml/typed/autoeach report their checkpointand
explicit model='…'as the reason, and--model autoprintsEnglish Latin text, i.e. it isexactly the same as leaving the flag off.
What changed
laya/cli.py— amodel_name()type function on the flag:auto(any casing) returnsNone,which is what omitting the flag already means, otherwise
laya.router.normalise_namedecides.metavar="NAME"solaya --helpreads--model NAMEinstead of enumerating three options.normalise_nameis imported inside the function, matching how the rest of the CLI reaches therouter lazily. One example line added to the module docstring.
README.md— the### Command lineblock gainslaya "My payment failed twice" --model ml.tests/test_cli.py— 38 → 69 checks. The sweep importsDEFAULT_MODELSand_ALIASESfromcore rather than writing the names out, so a new checkpoint or alias is covered the day it lands
instead of exposing the same gap again. Also covers: canonicalisation per spelling, the four
rejected inputs, the error message still naming every option, and
run_cliforwarding —--model enreachesrouter.routeas"english",--predict --model MLreachesrouter.predictas"multilingual",--model autoreaches it asNone.Gates:
ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120clean,compileallclean,tests/test_cli.py69/69,tests/test_hooks_api.py131/131.Overlap
build_parserand checks totests/test_cli.py, sowhichever of the three lands last needs a rebase; they touch different
add_argumentlines andthe same file.
docs/cli-mcp.mdpage that is not merged yet and does not touchlaya/cli.py. If it ends up listing--model's accepted values, it should say aliases andcasing are honoured rather than enumerate the three names — that is the mistake this flag made.